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CONTROL OF SEQUENCE OF VIDEO MODIFYING OPERATIONS 

Background of the Invention 
The invention relates to modifying video. 

When a video program is created at a nonlinear video editor, analog video 
5 source materials are often digitized, compressed and stored on disc for later random 
access for use in creating a video program. Segments of the video, also referred to as 
"video clips," can be taken from different sources, and the beginning and end points can 
be trimmed as desired during editing. The order of clips is established in a time line to 
determine the sequence, and the position of a clip can be changed. Clips can be deleted 
10 or inserted as desired. During the editing process, the clips of compressed video can be 
randomly accessed, decompressed and viewed in real time. 

The source video can be modified by various "pixel manipulations," which are 
fine grain processes (such as scale, rotate, blur), and "tools," which are larger grain 
processes (such as digitizing, color correcting, add to timeline, layer with another clip 
1 5 and master out). 

Compositing involves use of fine grain processes. In compositing applications, 
it is known to use a process tree to specify a set of transformations, filters, and effects 
to be applied to one clip. A process tree uses a dataflow metaphor, and processes are 
connected together with no storage nodes in between: As used herein, "video 
20 editing" includes traditional editing operations, compositing operations, and tool 
operations. 

Summary of the Invention 

In one aspect, the invention features, in general, a method of modifying video 
that includes receiving a first video clip, modifying the first video clip by one or more 
25 video tools in sequence to create a second video clip that is modified with respect to the 
first video clip, and recording the sequence of tools and the associated tool parameters 
used to modify the first video clip. Each tool modifies a clip input to it according to 
tool parameters and user interaction to generate an output clip. 

In another aspect, the invention features, in general, a method of modifying 
30 source video that includes receiving a first video clip, modifying the first video clip by 
one or more operations (e.g., bit manipulations and/or tools) in sequence to create a 
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second video clip that is modified with respect to the first video clip, recording the 
sequence of operations used to modify the first video clip, and displaying a history 
graph of the operations used to modify the first video clip. 

Preferred embodiments of the invention may include one or more of the 
5 following features. The tool history graph can be used in workflow navigation. E.g., 
video tools displayed on the tool history graph can be used with a different parameter 
file. There can be an intermediate video clip that is generated by a first tool and input 
to a second tool, and the second video tool can be used with a different parameter file. 
The first video tool can be used with a different parameter file, and the tool history 
10 graph will be updated to show that the intermediate video clip and the second video clip 
are outdated. The first tool can then be run with the different parameter file to result in 
an updated intermediate video clip, and the second video tool can be run on the updated 
intermediate video clip to result in an updated second video clip. 

The tool histoiy graph can be used to perform a media management function, 
15 such as moving unnecessary backing media, recreating clip backing media, and 

redigitizing clip backing media. The tool history graph can also be used to modify a 
clip history to create workflow steps. E.g., the user can create additional tools and new 
clips on the history. The tool history graph can specify an external tool, and the system 
will export a clip to the external tool and import the resulting clip. The tool history 
20 graph can be used to locate a clip on the histoiy, and a tool can be added to modify the 
located clip. 

In another aspect, the invention features, in general, a method of controlling 
media workflow by selecting a sequence of one or more operations to be used to 
modify a first video clip to result in a second video clip, displaying a graph of the 
25 operations, and thereafter modifying the first video clip by the selected and displayed 
operations. 

In preferred embodiments the operations include tools. A control decision can 
be used determine which of a plurality of possible tools could be used to modify the 
first video clip, and the display could include a control block showing the first clip as 
30 an input, and a plurality of workflow branches as output, each having a respective tool 
and a respective output clip. The control decision can be based on an if?then/else 
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decision. The control decision can be based on a selection control input that indicates 
one of the possible tools and associated workflow branch. There can be a plurality of 
clips as possible inputs, and the control decision can also determine which of the first 
clips is input, and the displaying would include the plurality of first clips. 

5 Embodiments of the invention may include one or more of the following 

advantages. The history display feature usefully permits one to access and visualize the 
history of a particular program clip. It also permits one to access earlier digitized 
versions of the clip without the need to reaccess the original source material. One can 
simply and quickly specify a new processing scheme or create a modified processing 
1 0 scheme for a clip by specifying new tools on the tool history diagram. 

Other advantages and features of the invention will be apparent from the 
following description of a preferred embodiment thereof and from the claims. 



Description of Drawings 
1 5 Fig. 1 is a block diagram of a nonlinear video editing system. 

Fig. 2 is a more detailed block diagram of some of the components of the Fig. 1 

system. 

Fig. 3 shows a time line and editing control box used on the display of the Fig. 1 

system. 

20 Fig. 4 shows a video editing software application and storage for clips and 

media. 

Fig. 5 shows a standard unit of work flow used in workflow recording and 
display in the Fig. 1 system. 

Fig. 6 shows an exemplary clip history displayed on the Fig. 1 system. 

25 Fig. 7 shows a standard unit of workflow including an if-then/else control block. 

Fig. 8 shows a standard unit of workflow including a control block control by a 
selection input. 
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Description of Particular Embodiments 

Referring to Fig. 1, nonlinear video editing system 10 includes host computer 
12 having monitor 14, user input devices 16 (e.g., keyboard and mouse), video editing 
5 software 1 8, video editing peripheral board 20 in an expansion slot of computer 12, 
video tape recorder (VTR) 24, and mass storage disc 26. Referring to Fig. 2, host 
computer 12 includes CPU 30, PCI bus 32, and input/output ports 34 for connection to 
monitor 14 and user input devices 16. Peripheral board 20 is connected to PCI bus 32. 
Board 20 includes coder/decoder (codec) 34, video input/output (I/O) port 36 for source 
10 video, color corrector 38 and busmaster circuit 40, which transfers data to and from PCI 
bus 32. Board 20 also has audio I/O ports, other video processing circuits, buffers, and 
hardware tools 102 to carry out various video editing and data transfer functions. The 
invention can also be used with software tools and third party hardware and software 
tools. 

15 Referring to Fig. 3, during editing a user can access various control windows on 

monitor 14 to carry out the editing functions. The user can place small picture icons 48 
and associated clip names identifying particular video clips in bin window 50, edit the 
beginning and end points of a particular video clip in edit window 52, and place a video 
clip in the program at time line 54. 

20 Each time that video material is accessed or modified in system 10 at a clip 

level, a software application program keeps track of the history of processing of a video 
clip and its relation to prior' or subsequent video clips. A representation of the history, 
called a tool history graph, can then be created and displayed at any time in tool history 
window 56, shown in Fig. 6 

25 In creating the tool history graphs, system 10 employs (1) clips, (2) clip history 

information, (3) used-on lists, (4) parameter files, and (5) tools. These objects are 
defined and described as follows: 
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1. Clip 

A clip is a unit of multimedia, usually video, with at least the following 
attributes: 

5 a well defined start and end time (in-out), and 

a description of the location in the media storage pool where the clip baclcing 
media is located. 

The clip may or may not have actual backing media. Users of nonlinear video 
editors typically manage hundreds to thousands of clips for a given project. 

10 

2. Clip History Information 
Clip history information includes: 

the name of the process (tool) that created the clip, 

the names of all of the input and output clips in the history of the generation of 
15 the clip, and 

parameter information such as a file name and a tool-dependent format for the 
information in the file. 

Clip history information can also include additional attributes such as name, 
create date, and media type. 

20 

3. Used-on List 

A used-on list is associated with a particular clip and is a list of processes that 
use the clip. This list is the forward link in the tool history graph; the clip history 
information (2. above) is the backward link. For a given clip, either the tool history 
25 information (back link), or the used-on-list (forward link), or both will be stored with 
the clip attributes in a clip store. 

5 
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4. Parameter Files 

Parameter files are auxiliary input used by a process (tool). The parameter file 
data does not necessarily have to be stored as a separate file so long as the parameters 
5 can be saved to and restored from disk in some form. Examples of parameter file data 
include scalar data such as clip background color, a motion slow down - speed up 
amount, the key color and acceptable range for a chroma keying operation. Parameter 
file data can also contain time varying data such as key framed position data and color 
values. In general, a parameter file contains all of the data needed by a process (tool) to 
1 0 operate on an input clip to produce a specific output clip. The parameter file 

encapsulates the use of the tool by the user. The content of a parameter file is the direct 
or indirect result of user input. 

5. Tool 

15 A tool is a discreet transformation performed on a clip that results in a new clip 

being produced. A tool is usually designed to perform one type of nonlinear video 
editing and/or compositing operation such as motion effect, keying, painting, etc. A 
tool is a unit of workflow as compared to a unit of pixel manipulation. A tool is most 
often started and provided an input clip on which the tool will work. The output of a 

20 tool is not directly connected to another tool; it is always connected to a clip. Examples 
of tools include: input digitize tool, matte tool, master out tool, paint tool, color 
corrector, keyer (e.g., chroma and/or luminance key process), motion effects tools (e.g., 
fast or slow motion), timeline tool (which can do editing (horizontal) and layering 
(vertical) operations), motion tracking and third party applications. 

25 Referring to Fig. 4, there is shown nonlinear editor application 1 04 (used in 

system 10) with clip storage area 106 (also called "clip store") for clip data (described 
as 1 . above) and media storage area 108 (also called "media store") for the backing 
media files to which the clip data refer. Note that a clip can exist without actual media 
in the media storage area 108. The clip history (described as 2. above) is also stored in 

30 clip storage area 108. 
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Referring to Fig. 5, a standard unit of work flow can be represented by an input 
clip CI (1 10), an output clip C2 (1 12), a tool 1 14, and parameters file 1 16. The input 
clip CI is usually kept in the clip storage area 106 after tool 1 14 is finished. The media 
backing for clip CI, however, may be deleted. The output clip C2 almost always has 
5 media backing in the media storage area 108. The parameters file 116 serves as input 
to the tool 1 14 and as a place to save all user input on tool exit. Parameters file 116 
may be empty on tool start-up, and contain a list of all of the user commands on exit. 
At a minimum, parameters file 116 must contain the information necessary to produce 
the output clip, C2, at a later stage, given CI and a copy of parameter file 116. In the 
1 0 aggregate history of all clips, a single clip can feed more than one process. That is, 
work flow can fork into two paths at a clip node in the graph. A tool 114 can accept 
more than one clip as input and produce more than one clip as output. 

The processing involved in implementing a tool history graph includes (1) 
recording the history of a clip, (2) displaying the history of one or more clips in an 

1 5 interactive graph, and (3) allowing user interaction with that graph to, e.g., accomplish 
(a) workflow navigation, i.e., random access to steps performed over the life of a 
project, (b) media management, e.g., removing unwanted media, regenerating media, 
redigitizing media, and other tedious media management functions, and (c) workflow 
step creation, i.e., outlining the steps to be taken during an editing session. For 

20 example, to create a workflow step, the user can digitize one clip, color correct it, add it 
to a timeline, and then take that sequence of events and apply it to a whole set of clips. 
The steps just mentioned for implementing a tool history graph are described in the 
following sections, which also include a section (d) dealing with further workflow 
controls. 

25 

1 . Recording History 

As the user creates a clip within application 104 (Fig. 4), a description of the 
clip is created and maintained in the clip store 1 06, which is where all clips and clip 
history information reside. Clips are created as a result of running a tool. The input 

7 
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digitize tool (which imports raw video into the application) is the most common creator 
of clips. The history of a digitized clip will indicate the digitize tool as the creating 
process, and, rather than an input clip, the history will have a tape name as input. 
Similarly the master out tool will write to a tape (usually) as output and have clips as 
5 inputs. 



2. Displaying a Clip History 

At one level, a tool history graph can be simply used to display the history of a 
clip or clips without user interaction with the graph (other than pan/zoom and similar 

10 presentation operations.) The user can display the history of one or more clips 

graphically or textually in purpose designed windows. A graphic representation can 
appear as in Fig. 5 (discussed above) for one a one-tool history or Fig. 6 (discussed 
below) for a history involving many tools, A text based clip history display lists 
hierarchically the clip name, the name of the creating process, and the names of the 

15 input clips and parameters. Then a similar history is also prepared for each input clip 
of that tool. The result is a text tree listing the complete ancestry of the initial clip in 
outline format. The user can specify how many links backward to trace the clip 
ancestry. For example, the user could display the history of the final output program 
clip and request that all history paths leading to that final clip be displayed back to 

20 where each initial clip was digitized. 



3 . User/Graph Interaction 

A. Navigating Clip History - Workflow Navigation 

At a further level of tool history graph usage, the user interacts with the graph to 
25 navigate the history of a clip to access a previously completed step to re-execute a 
process (tool). Tool re-launch is implemented by a user selecting (e.g., by clicking 
on) a tool in the tool history graph, which causes that tool to be launched with the 
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appropriate clip and parameter input(s) such that the state of the tool is the same as 
when it was last executed for that clip and parameters. The user can change one or 
more of the tool's parameters to change the tool's result. This will outdate (make 
obsolete) the output clip(s) of the tool as well as any subsequent clips in the history 
5 that depend on those output clips. The outdated nature of the clips is graphically 
indicated (e.g., different color, lines or shading) on the tool history graph. If 
desired, the user can select that all downstream processes are automatically run in 
order to update each affected subsequent clip, resulting in a great savings in time 
over manual operation. 

10 

B. Media Management 

The tool history graph can also be used to perform at least the following media 
management tasks: 

i. move unnecessary clip backing media offline, 

15 ii. recreate or redigitize clip backing media files, possibly at 

different resolutions, 

iii. cut, copy and paste clips, and 

iv. consolidate backing material of a selected set of clips. 



20 C. Modifying Clip History - Creating Workflow Steps 

A further, powerful use of the tool history graph is defining sequences of 
workflow steps. This is accomplished by dragging tool icons 118 (Fig. 6) into the 
graph display, dragging clip icons 48 (Fig. 3) from bin window 50 or other places, and 
connecting them together to form a sub-graph (i.e., new workflow sequence.) Users can 

9 
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access existing clips or tools, or create a new clip and use an unparameterized tool. 
Some of the workflow operations that can be performed are: 

V 

i. copy an existing sequence of steps, paste it as a new sequence, 
and connect a new input clip to the sub-graph, 

5 ii. convert media by dragging an existing input clip (having RGB 

color) icon, a convert-RGB-to-YUV tool icon, and output clip (having YUV color) into 
the graph and connecting them together and executing the workflow, 

iii. setup for a batch digitize by starting with a timeline program 
clip, selecting all initial input clips (ancestor clips whose creating process is the 

10 digitizer), and specifying a re-digitize operation, 

iv. export clips (similar to the convert media example above, 
with the tool exporting media and having no clip output, per se), 

v. call an external tool, which causes an input clip to be directed 
to an external tool for processing and return, and 

15 vi. fork project development to. cause a particular clip or clips to 

branch into more than one tool downstream in the workflow. 

Fork project management can be used for "versioning." The user can add a set 
of clips into a timeline, add tool icons to the graph and produce an output. The user can 
then select all of the steps back to the set of clips, copy that sequence, paste it as a new 
20 sequence, and modify the parameters controlling the tool (or use different tools) to 
produce a different look. The user can thus very easily try different approaches to 
content design and delete the versions (media and process parameters) that are not 
going to be used. 

Fig. 6 shows an exemplary tool history graph that displays the following 
25 sequence of steps: 
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1 . The user digitizes video tape 120 at tool 122 into a clip, clip 1 (124) using user 
parameters 126. 

2. Then the user places clip 1 (124) (or a copy of clip 1) directly in the timeline 
126; this is indicated by arrow 128. 

5 3. The user also takes clip 1 (124) (or a copy of clip 1) and runs it through color 

corrector 130, which produces a color-corrected clip, clip 2 (132) using user parameters 
134. 

4. The user then puts clip 2 (132) in timeline 126, where the two clips are edited 
together according to user parameters 136. 

10 5. The user then renders timeline 126 to produce program clip 138. 

If the user subsequently decides that he or she is not satisfied with the color 
correction, instead of making further modifications to the color corrected version 132, 
or reaccessing the material from video tape 120, the user can access the digital version, 
clip 1 (124), as it existed prior to color correction. The clip history feature of system 
15 10 usefully permits one to access and visualize the history of a particular program clip. 
It also permits one to access earlier digitized versions of the clip without the need to 
reaccess the original source material. 

System 10 also permits one to specify a new processing scheme or to create a modified 
processing, scheme for a clip using tool history palette 140, which includes tool icons 

20 1 18 to specify, e.g., a digitizer, color corrector, keyer (e.g., chroma luminance key 

process), motion effects (e.g., fast or slow motion) and other tools. The user accesses a 
tool by clicking on it and dragging it to the desired location in the processing diagram, 
and can insert the tool as an additional process or to replace a displayed process. The 
user can also add a new source video or add a new clip (from the bin window 50 or 

25 identified by name). When a new clip is inserted part way through a process, the prior 
steps and image source will be replaced with the prior history for the selected clip. The 
subsequent clips will also be replaced with new clip identifiers for new clips to be 
created. When the user is satisfied, he or she can enable system 10 to then create the 
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new clips (the ones that come after the substituted clip in the process flow) by running 
indicated tools on the substituted clip, unless the specified tools have already been run 
for the substituted clip. 

The tool history graph can also be modified to record and display pixel 
5 manipulations. In this case, instead of tool icons and tool parameter files, or in addition 
to tool icons, there would be pixel manipulation icons that indicate the operation (e.g., 
scale, rotate, blur, layer with another clip) and associated pixel manipulation parameter 
files specifying the operation. 

10 D. Further Workflow Control 

The tool history graph as described herein can be used in the absence of a "history" 
to define and create workflow in the first instance by entering icons indicating clips and 
tools onto a blank graph. 

The graph can, as a part of forked project management, include control constructs 

15 in the diagram to provide different possible flowpaths on the graph and control blocks 
used to determine the flow of control on the graph based upon inputs or predefined 
conditions. In particular, if/then/else control blocks can be added to the diagram as 
shown for the control block and workflow units shown in Fig. 7. The input clip 150 is 
shown connected to control block 152, which in this example determines which 

20 workflow branch to use in processing clip 150 based upon whether clip 150 is the PAL 
video type or not. If clip 150 is PAL, then tool 154 is used to generate clip 156. If clip 
150 is not PAL, then tool 158 is used to generate clip 160. In addition to media type, 
any other media attribute, or any other variable in general, can be used to determine the 
workflow branch at control block 1 52. 

25 Another example of a control block, a switch tree, is shown in Fig. 8. In this case, 
clip 162 is the input clip, and switch tree 164 has five possible output paths 1-5, 
employing respective tools 166-174 to generate respective output clips 176-184. 
Switch tree 1 64 selects one of the five output paths depending on the value of input S 
being 1-5. It is also possible for a switch control block to have multiple inputs (each 

30 
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with a respective input clip) and multiple outputs, with one or more control inputs used 
to select the input clip and output workflow path. The use of control blocks in the tool 
history graph workflow diagram provides a large number of possibilities for efficiently 
creating media and doing batch processing. 
5 Other embodiments of the invention are within the scope of the following claims. 

What is claimed is: 
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1 . A method of modifying video comprising 
receiving a first video clip, 

modifying said first video clip by one or more video tools in sequence to create a 
5 second video clip that is modified with respect to said first video clip, each said tool 
modifying a clip input to it according to tool parameters to generate an output clip, and 

recording the sequence of tools and the associated too] parameters used to modify 
said first video clip. 

2. A method of modifying video comprising 
1 0 receiving a first video clip, 

modifying said first video clip by one or more operations in sequence to create a 
second video clip that is modified with respect to said first video clip, 

recording the sequence of operations used to modify said first video clip, and 
displaying a histoiy graph of said operations used to modify said first video clip. 
15 3. The method of claim 2 wherein said operations include tools, each said tool 

modifying a clip input to it according to tool parameters to generate an output clip. 

4. The method of claim 2 or 3 wherein said operations include pixel manipulations 

5. The method of claim 1 further comprising displaying a history of said tools 
used to modify said first video clip. 

20 6. The method of claim 1 or 5 further comprising repeating use of a said video tool 
with a different parameter file. 

7. The method of claim 1 or 5 wherein there are first and second said tools and 
an intermediate video clip that is generated by said first tool and input to said second 
tool, and further comprising repeating use of said second video tool with a different 

25 parameter file. 

8. The method of claim 5 wherein there are first and second said tools and an 
intermediate video clip that is generated by said first tool and input to said second tool, 
and further comprising changing said display to show a use of said first video tool with 
a different parameter file and to show that said intermediate video clip and second 

30 video clip are outdated. 

9. The method of claim 8 further comprising repeating said use of said first tool 
with said different parameter file to result in an updated intermediate video clip and 



14 



WO 01/60060 



PCT/USO 1/04264 



repeating use of said second video tool on said updated intermediate video clip to result 
in an updated second video clip. 

10. The method of claim 2 or 5 further comprising performing a media 
management function based upon said displaying of said history. 
5 11. The method of claim 2 or 5 further comprising performing a media 

management function based upon said displaying of said history, and wherein said 
media management function is a function selected from the list of removing 
unnecessary backing media, recreating clip backing media, redigitizing clip backing 
media, and cutting, copying and/or pasting clips. 

10 12 - The method of claim 5 further comprising creating additional tools and/or 

new clips on said history. 

13. The method of claim 5 wherein a said tool is an external tool, and further 
comprising exporting to said external tool and importing the resulting clip. 

14. The method of claim 5 further comprising selecting a clip on said history 
1 5 and using a tool to modify the located clip; 

1 5. Computer apparatus for modifying video comprising 
a processor, 

a user input device, 
a video editing application, 
20 a clip store that is accessible by said application and stores clips, 

a backing media store that is accessible by said application and stores media 
specified by said clips, 

tools for performing operations on clips, and 

parameter files specifying parameters for said tools, each said tool modifying a 
25 clip input to it according to said tool parameters to generate an output clip, said 

application recording the sequence of tools and the associated tool parameters used to 
modify a first video clip into a second video clip. 

16. Computer apparatus for modifying video comprising 
a processor, 

30 a user input device, 

a video editing application, 

a clip store that is accessible by said application and stores clips, 
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a backing media store that is accessible by said application and stores media 
specified by said clips, 

said application recording the sequence of operations used to modify a 
first video clip, and 

5 a display that shows a history graph of the operations used to modify said first 

video clip. 

17. The apparatus of claim 16 further comprising tools and wherein said 
display displays operations that include said tools. 

18. The apparatus of claim 16 or 17 wherein said display displays operations 
10 that include pixel manipulations. 

19. The apparatus of claim 15 further comprising a display that shows a history 
graph of said tools used to modify said first video clip. 

20. The apparatus of claim 15 or 19 wherein a said video tool can be used with a 
different parameter file. ^ 
15 21. The apparatus of claim 15 or 19 wherein there are first and second said tools 

and an intermediate video clip that is generated by said first tool and input to said * ~ 

second tool, and wherein use of said second video tool is repeated with a different 
parameter file. 

22. The apparatus of claim 19 wherein there are first and second said tools and - 
20 an intermediate video clip that is generated by said first tool and input to said second ":' 

tool, and wherein said display is changed to show a use of said first video tool with a 
different parameter file and to show that said intermediate video clip and second video 
clip are outdated. 

23. The apparatus of claim 22 wherein use of said first tool is repeated with said 
25 different parameter file to result in an updated intermediate video clip and use of said 

second video tool is repeated on said updated intermediate video clip to result in an 
updated second video clip. 

24. The apparatus of claim 19 wherein a said tool is an external tool. 

25. A method of modifying video comprising 

30 selecting a sequence of one or more tools to be used to modify a first video clip 

to create a second video clip that is modified with respect to said first video clip, 

displaying a graph of said one or more tools used to modify said first video clip 
to create said second video clip, and 
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thereafter modifying said first video clip by said one or more tools in sequence. 
26. A method of modifying video comprising 

selecting a sequence of one or more operations to be used to modify a first 
video clip to create a second video clip that is modified with respect to said first video 
5 clip, 

displaying a graph of said one or more operations used to modify said first 
video clip to create said second video clip, and 

thereafter modifying said first video clip by said one or more operations in 
sequence, 

1 0 wherein said selecting includes selecting a control decision to determine which 

of a plurality of possible operations will be used to modify said first video clip, and said 
displaying includes displaying a control block showing said first clip as an input, mid a 
plurality of workflow branches as output, each having a said possible operation and a 
respective output clip. 

15 27. The method of claim 26 wherein said operations include pixel 

manipulations. 

28. The method of claim 25 wherein said selecting includes selecting a control 
decision to determine which of a plurality of possible tools will be used to modify said 
first video clip, and said displaying includes displaying a control block showing said 

20 first clip as an input, and a plurality of workflow branches as output, each having a said 
possible tool and a respective output clip. 

29. The method of claim 26 or 28 wherein said control decision is based on an 
if/then/else decision. 

30. The method of claim 26 or 28 wherein said control decision is based on an 
25 a selection control input that indicates a said possible tool and associated workflow 

branch. 



3 1 . The method of claim 26 or 28 wherein there are a plurality of said first clips 
as possible inputs, and said control decision also determines which said first clip is 
input, and said displaying includes displaying said plurality of first clip. 

30 
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